Skip to content

Comments

feat: OpenAI-compatible gateway endpoints + Bedrock streaming#9

Closed
shadmoney wants to merge 3 commits intoopenagen:mainfrom
SainIndustries:feat/openai-compat
Closed

feat: OpenAI-compatible gateway endpoints + Bedrock streaming#9
shadmoney wants to merge 3 commits intoopenagen:mainfrom
SainIndustries:feat/openai-compat

Conversation

@shadmoney
Copy link

Summary

  • OpenAI-compatible /v1/chat/completions — supports streaming (SSE) and non-streaming responses, bearer token auth, rate limiting, and a 512KB body limit for large chat histories. Falls back to single-chunk SSE when the provider doesn't natively support streaming.
  • OpenAI-compatible /v1/models — returns the gateway's configured model in standard OpenAI format.
  • Bedrock ConverseStream support — implements stream_chat_with_system() for the Bedrock provider using the /converse-stream endpoint with SigV4 signing and a minimal AWS binary event-stream parser (no new crate dependencies).
  • Test fix — adds missing skills_prompt_mode argument in build_system_prompt test that was broken after a recent signature change.

Motivation

These changes enable standard OpenAI client libraries (Python openai, TypeScript openai, LangChain, etc.) to interact with ZeroClaw's gateway directly, which is the main integration path for applications migrating from OpenAI-compatible gateways. Bedrock streaming enables real-time token-by-token responses for AWS Bedrock users.

Test plan

  • 8 new unit tests for OpenAI-compat request/response serialization
  • 10 new unit tests for Bedrock streaming (URL formatting, binary event-stream parsing, deserialization)
  • All 54 gateway tests pass
  • All 48 Bedrock tests pass
  • cargo build succeeds
  • Manual testing with curl against /v1/chat/completions and /v1/models

🤖 Generated with Claude Code

shadmoney and others added 3 commits February 22, 2026 19:25
The test was not updated when the function signature changed to include
the skills_prompt_mode parameter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ints

Add an OpenAI-compatible API surface to the gateway so that standard
OpenAI client libraries can interact with ZeroClaw directly.

Endpoints:
- POST /v1/chat/completions — supports both streaming (SSE) and
  non-streaming responses, bearer token auth, rate limiting
- GET /v1/models — returns the gateway's configured model

The chat completions endpoint accepts the standard OpenAI request format
(model, messages, temperature, stream) and returns responses in the
OpenAI envelope format. Streaming uses SSE with delta chunks and a
[DONE] sentinel. A 512KB body limit is applied (vs 64KB default) since
chat histories can be large.

When the underlying provider doesn't support native streaming, the
handler falls back to wrapping the non-streaming response in a single
SSE chunk for transparent compatibility.

Includes 8 unit tests for request/response serialization.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement the streaming provider trait methods for Bedrock, enabling
real-time token-by-token responses via the ConverseStream endpoint.

Key implementation details:
- Uses /model/{id}/converse-stream endpoint with SigV4 signing
- Parses AWS binary event-stream format (application/vnd.amazon.eventstream)
  with a minimal parser (~60 lines) — no new crate dependencies needed
- Handles contentBlockDelta events for text extraction, plus error and
  exception events
- Uses mpsc channel + stream::unfold pattern (matching compatible.rs)
- Clones credentials for async task ownership

The binary event-stream parser extracts frame lengths, header sections
(looking for :event-type), and payload bytes. CRC validation is skipped
since TLS already provides integrity guarantees.

Includes 10 new tests for URL formatting, binary parsing, and
deserialization.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link

Thanks for contributing to ZeroClaw.

For faster review, please ensure:

  • PR template sections are fully completed
  • cargo fmt --all -- --check, cargo clippy --all-targets -- -D warnings, and cargo test are included
  • If automation/agents were used heavily, add brief workflow notes
  • Scope is focused (prefer one concern per PR)

See CONTRIBUTING.md and docs/pr-workflow.md for full collaboration rules.

@github-actions
Copy link

PR intake checks found warnings (non-blocking)

Fast safe checks found advisory issues. CI lint/test/build gates still enforce merge quality.

  • Missing required PR template sections: ## Validation Evidence (required), ## Security Impact (required), ## Privacy and Data Hygiene (required), ## Rollback Plan (required)
  • Incomplete required PR template fields: summary problem, summary why it matters, summary what changed, validation commands, security risk/mitigation, privacy status, rollback plan

Action items:

  1. Complete required PR template sections/fields.
  2. Remove tabs, trailing whitespace, and merge conflict markers from added lines.
  3. Re-run local checks before pushing:
    • ./scripts/ci/rust_quality_gate.sh
    • ./scripts/ci/rust_strict_delta_gate.sh
    • ./scripts/ci/docs_quality_gate.sh

Run logs: https://github.com/openagen/zeroclaw/actions/runs/22288816257

Detected blocking line issues (sample):

  • none

Detected advisory line issues (sample):

  • none

@github-actions github-actions bot added agent Auto scope: src/agent/** changed. provider Auto scope: src/providers/** changed. dependencies Auto scope: dependency manifest/lock/policy changed. gateway Auto scope: src/gateway/** changed. size: XL Auto size: >1000 non-doc changed lines. risk: high Auto risk: security/runtime/gateway/tools/workflows. agent: loop Auto module: agent/loop changed. gateway: openai_compat Auto module: gateway/openai_compat changed. and removed agent Auto scope: src/agent/** changed. gateway Auto scope: src/gateway/** changed. labels Feb 23, 2026
@shadmoney
Copy link
Author

Opened against wrong upstream — closing.

@shadmoney shadmoney closed this Feb 23, 2026
@shadmoney shadmoney deleted the feat/openai-compat branch February 23, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent: loop Auto module: agent/loop changed. dependencies Auto scope: dependency manifest/lock/policy changed. gateway: openai_compat Auto module: gateway/openai_compat changed. provider Auto scope: src/providers/** changed. risk: high Auto risk: security/runtime/gateway/tools/workflows. size: XL Auto size: >1000 non-doc changed lines.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant